A New Nonextensive Entropy 
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We propose a new way of defining entropy of a system, which gives a general form which may 
be nonextensive as Tsallis entropy, but is linearly dependent on component entropies, like Renyi 
entropy, which is extensive. This entropy has a conceptually novel but simple origin and is mathe- 
matically easy to define by a very simple expression, though the probability distribution resulting 
from optimizing it gives rather complex, which is compared numerically with the other entropies. 
It may, therefore, appear as the right candidate in a physical situation where the probability distri- 
bution does not suit any of the previously defined forms. 
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I. INTRODUCTION 

Entropy is a measure of disorder or randomness, and 
assumes its maximal value when a system can be in a 
number of states randomly with equal probability, and 
is minimally zero when the system is in a given state, 
with no uncertainty in its description. Apart from this 
common feature shared by all definitions of entropy at 
two ends of the scale, variations are possible in particu- 
larizing the functional form in between [l| . They lead to 
different forms of the probability distributions for states 
with different energies or some other conserved attribute. 
Some turn up as extensive, where the entropy of a com- 
bination of systems is simply the sum of the entropies 
of the systems, as in the classical case of the Shannon 
form, while others can be defined to be not so. Renyi 
entropy 2] is different from Shannon, yet extensive, and 
hence the Shannon form is not unique with respect to the 
property of extensivity. 

Tsallis entropy 3? ] has attracted a lot of attention in 
recent years, not only on account of its conceptual and 
theoretical novelty, but also because it can be shown in 
specific physical cases |a H 13 to be the relevant form 
where nonextensivity is expected on account of the inter- 
action of the combined subsystems. In the proper lim- 
iting case it reduces to the standard Shannon entropy, 
indicating the consistency of the concept. 

In this paper, however, we shall introduce entropy from 
a new perspective, which too will bear semblance to the 
normal form in the limit. We shall first present the ratio- 
nale for this new definition and compare it briefly with 
the forms already being used. Then we shall find the form 



of the probability distribution for this entropy, which we 
shall henceforth call s-entropy, as it will be seen to be 
related to the concept of rescaling of the phase space. 



II. DEFINING THE NEW ENTROPY 

Let us consider a register of only one letter. Let Pi be 
the set of probabilities for each of the N letters Ai that 
can occupy this position. We are here using the language 
of information theory, as used for example, in the Shan- 
non Coding Theorem, though it is trivially extensible to 
states j of a single state of an ensemble where the indi- 
vidual systems can be in any N states with probabilities 
Pi- 

Let us now consider a small deformation of the register 
to a new size so that it can accommodate q = 1 + Aq 
letters. The probability that the whole new phase space 
is occupied by the letter pi is now p\ by the corresponding 
AND operation and hence the probability that the new 
deformed cell is occupied by any of the pure letters Ai is 



#(«) = £p? 



(i) 



For q > 1 this would give a shortfall from the original 
total probability of unity for q = 1 . It is obvious that the 
shortfall, which we denote by 



M(g) = l-5>* 



(2) 
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represents the total probability that the mixed cell has 
a mixture of Ai and some other A, fractionally, since 
the total probability that the cell is occupied by one or 



more (fractional included) letters must be unity. Hence 
the mixing probability M(q) is actually a measure of the 
disorder introduced by increasing the cell scale from unity 
to 1 + Aq. 

The introduction of fractional values of cell numbers 
can be taken in the same spirit as defining the fractal 
(Hausdorff) dimensions of curves, and in complex sys- 
tems there have been studies of diffusion |9j and per- 
colation in complex systems with effectively fractional 
dimensions for fluids where the special geometric con- 
straints translate into a change in the dimension of the 
corresponding space to an apparently nonintuitive frac- 
tional dimension. In coding theories for optimal trans- 
mission of information |l(l j. we come across Huffmann 
coding, where the optimum alphabet size may be for- 
mally a fraction, though for practical purposes it may be 
changed to the nearest higher integer. In probabilistic 
optimization, we may therefore consider a fractional size 
of the registrar, or equivalently, an integral number of 
cells in the registrar with fractional sized cells to accom- 
modate a given amount of information. Probabilistic op- 
timization in place of the deterministic parameterization 
of classical Shannon information theory |l(j becomes in- 
evitable in quantum computing contexts, and hence our 
use of the fractional cell sizes may be a classical precursor 
of the inevitable departure from stringent Shannon-type 
concepts. 

For an alphabet of mletters we define the entropy from 
the information content of the registrar by 



in 



S(q)Aq 



AM(q+Aq)-M(q)) 



(3) 



so that the entropy indicates an effective change in the 
mixing probability due to an infinitesimal change in the 
cell-size of the registrar. 

This leads to 



where the generalized q-logarithm is defined as 



Log qPi = l-p < f 1] /{l-q) 



(8) 



The expectation value is defined in terms of the simple 
probability distribution 



< o>=J2p*°* 



(9) 



In our case we define the expectation value with re- 
spect to the deformed probability corresponding to the 
extended cell, while keeping the usual logarithm 



with 



S s (q) =< logp> q 



<o> q =J2 P io t 



(10) 



(11) 



In the limit q — > 1 Log q approaches the normal loga- 
rithm, and hence Tsallis entropy coincides with Shannon 
entropy and also as p\ —>■ pi we too get the normal Shan- 
non entropy. 

The Renyi entropy is defined by 



S*(g)=bg(5>?)/(l-«) 



(12) 



Like Shannon entropy this one is also extensive, i.e. 
simply additive for two subsystems for an y v alue of q. 
To get Shannon entropy uniquely one needs [ll| a slightly 
different formulation of the extensivity axiom 



S(q) = dM(q)/dq 



In other words 



S(q) = -^pjlogpi 



(4) 



(5) 



We have some material at the other place. This 
differential form is analogous to but different from the 
Tsallis form 



S T (q) = -J2(l-P?)/(l-<l) 



(6) 



•Si+2 = Si +y^^pijS 2 (i), 



(13) 



where S^j) is the entropy of subsystem 2 given sub- 
system 1 is in state i. 



III. PROBABILITY DISTRIBUTION FOR THE 
NEW ENTROPY 

The pi can be obtained in terms of the energy of the 
states, or possibly also other criteria in the usual way by 
maximizing the entropy with constraints 



where there is an apparent singularity at q = 1 which 
is the Shannon limit. The difference between the Tsallis 
expression and ours becomes clearer if we express entropy 
as the expectation value of the (generalized or ordinary) 
logarithm. 



Sr(q) =< Log q p> 



(7) 



and 



X>-1 = 



J2P^E^-U = Q 



(14) 



(15) 



The solution of the optimization equation gives for en- 
ergy Ei the probability pi 

where 

z=-e ( - q - 1 ^ q (a + bE)(q-l)/q (17) 

and W(z) is the Lambert function defined by 
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z = we w (18) 

Here a and b are constants coming from the Lagrange's 
multipliers for the two constraints and are related to the 
overall normalization and to the relative scale of energy, 
i.e. to temperature ( l/(fcT)) as in the Shannon case 
where we get the Gibbs expression for pi . In the Tsallis 
case pi has the well-known value 



Pi = (a + &(g-l)£0 1/(1_,) (19) 

which is easily seen to reduce to Shannon form for q — ► 
1. 

After some algebra it can be shown that this form re- 
duces to the Shannon form for q — > 1. 

The nonextensivity of Tsallis entropy is seen easily by 
expanding 



s T i+2 = -j2piPj( 1 -Pi 9 pj 9 )/(i - if 



^S 1 1 + S 1 2 + {l-q)S 1 1 S 1 2 



(20) 



(21) 



For Renyi entropy we have the simple additive relation 



qR qR i qR 

<J 1+2 — <-> 1 + <-> 2 



(22) 



In case of the new entropy 



S s 1+2 = S\ + S s 2 + M 2 {q)S s 1 + Mi(ff)5' 2 (23) 

where the M a are the mixing probability of states for 
subsystem a as defined in Eq.2. 



FIG. 1: Comparison of the pdf for the new entropy for values 
of q = 1, 1.1, 1.2 and 1.3. The solid line is for q — 1, i.e. the 
Gibbs exponential distribution and the lines are in the order 
of q 



standard value of q = 1 can cause a quite discernible 
change in the pdf and should be observable in experi- 
mental contexts fairly easily . At q — 1.3, the shape is 
almost linear. 

In Fig. 2 and Fig. 3 we show the comparison of Tsallis 
pdf and the pdf for the new entropy for the same values 
of q, 1.1 in the former and 1.3 in the latter. We no- 
tice that for larger q values the new entropy gives much 
stiffer probability functions departing substantially from 
the Tsallis pdf's. 
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FIG. 2: Comparison of pdf's for Tsallis nonextensive entropy 
and the new entropy presented here, for q — 1.1 



CONCLUSIONS 



IV. NUMERICAL COMPARISON 

In Fig.l we show the variation of the probability func- 
tion for different E at different q values. 

We note that the pdf drops increasingly rapidly for 
higher values of q, and is quite different in shape and in 
magnitude at high energy values from the Gibbs expo- 
nential distribution. A variation of even 10% from the 



We see that the new entropy presented here based on 
the simple concept of the amount of mixing of states 
freedom introduced per unit cell of phase space leads to 
a nonextensive form different from any of the presently 
studied entropies. It leads to a complicated, but still 
integrable form of the pdf which departs substantially 
from Tsallis entropy. This entropy is also nonextensive 
in a fashion different from Tsallis entropy, though like 
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Tsallis it too becomes extensive trivially in the limit q 
1, as expected. 
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FIG. 3: The same as Fig. 2, but for a higher q — 1.3 



It would now be interesting to find a physical situation 
where such an entropy arises from first principles, though 
like some initial phenomenological studies of Tsallis en- 
tropy it can be also used as a parametrization scheme 
with q as a parameter to fit experimental data. The stiff- 
ness of any data may point to its preferability to Tsallis- 
type entropies. 
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